Search for: All records

We present a personalized, comprehensive eye-tracking solution based on tracking higher-order Purkinje images, suited explicitly for eyeglasses-style AR and VR displays. Existing eye-tracking systems for near-eye applications are typically designed to work for an on-axis configuration and rely on pupil center and corneal reflections (PCCR) to estimate gaze with an accuracy of only about 0.5°to 1°. These are often expensive, bulky in form factor, and fail to estimate monocular accommodation, which is crucial for focus adjustment within the AR glasses. Our system independently measures the binocular vergence and monocular accommodation using higher-order Purkinje reflections from the eye, extending the PCCR based methods. We demonstrate that these reflections are sensitive to both gaze rotation and lens accommodation and model the Purkinje images’ behavior in simulation. We also design and fabricate a user-customized eye tracker using cheap off-the-shelf cameras and LEDs. We use an end-to-end convolutional neural network (CNN) for calibrating the eye tracker for the individual user, allowing for robust and simultaneous estimation of vergence and accommodation. Experimental results show that our solution, specifically catering to individual users, outperforms state-of-the-art methods for vergence and depth estimation, achieving an accuracy of 0.3782°and 1.108 cm respectively. 

Abstract Approximately 10% of the world’s population is at risk of schistosomiasis, a disease of poverty caused by theSchistosomaparasite. To facilitate drug discovery for this complex flatworm, we developed an automated high-content screen to quantify the multidimensional responses ofSchistosoma mansonipost-infective larvae (somules) to chemical insult. We describe an integrated platform to process worms at scale, collect time-lapsed, bright-field images, segment highly variable and touching worms, and then store, visualize, and query dynamic phenotypes. To demonstrate the methodology, we treated somules with seven drugs that generated diverse responses and evaluated 45 static and kinetic response descriptors relative to concentration and time. For compound screening, we used the Mahalanobis distance to compare multidimensional phenotypic effects induced by 1323 approved drugs. Overall, we characterize both known anti-schistosomals and identify new bioactives. Apart from facilitating drug discovery, the multidimensional quantification provided by this platform will allow mapping of chemistry to phenotype.